Minimally invasive surgical techniques are aimed at reducing the amount of extraneous tissue that is damaged during diagnostic or surgical procedures, thereby reducing patient recovery time, discomfort, and deleterious side effects. As a consequence, the average length of a hospital stay for a standard surgery may be shortened significantly using minimally invasive surgical techniques. Also, patient recovery times, patient discomfort, surgical side effects, and time away from work may also be reduced with minimally invasive surgery.
A common form of minimally invasive surgery is endoscopy, and a common form of endoscopy is laparoscopy, which is minimally invasive inspection and surgery inside the abdominal cavity. In standard laparoscopic surgery, a patient's abdomen is insufflated with gas, and cannula sleeves are passed through small (approximately ½ inch or less) incisions to provide entry ports for laparoscopic surgical instruments.
The laparoscopic surgical instruments generally include a laparoscope or an endoscope (for viewing the surgical field), and working tools. The working tools are similar to those used in conventional (open) surgery, except that the working end or end effector of each tool is separated from its handle by an extension tube. As used herein, the term “end effector” means the actual working part of the surgical instrument and can include clamps, graspers, scissors, staplers, image capture lenses, and needle holders, for example.
To perform surgical procedures, the surgeon passes these working tools or instruments through the cannula sleeves to an internal surgical site and manipulates them from outside the abdomen. The surgeon views the procedure by means of a monitor that displays an image of the surgical site taken from the laparoscope. Similar endoscopic techniques are employed in, e.g., arthroscopy, retroperitoneoscopy, pelviscopy, nephroscopy, cystoscopy, cisternoscopy, sinoscopy, hysteroscopy, urethroscopy, and the like.
Minimally invasive telesurgical robotic systems are being developed to increase a surgeon's dexterity when working within an internal surgical site, as well as to allow a surgeon to operate on a patient from a remote location. In a telesurgery system, the surgeon is often provided with an image of the surgical site at a computer workstation. While viewing a three-dimensional image of the surgical site on a suitable viewer or display, the surgeon performs the surgical procedures on the patient by manipulating master input or control devices of the workstation. Each of the master input devices controls the motion of a servomechanically operated surgical instrument. During the surgical procedure, the telesurgical system can provide mechanical actuation and control of a variety of surgical instruments or tools having end effectors that perform various functions for the surgeon, e.g., holding or driving a needle, grasping a blood vessel, or dissecting tissue, or the like, in response to manipulation of the master input devices.
During the surgical procedure, however, the surgeon may manipulate the tool so that its end effector is moved outside of the endoscope's field of view, or the end effector may become difficult to see due to occlusion by fluids or other intervening objects. In such cases it would be useful to be able to provide assistance to the surgeon in locating and/or identifying the end effector on the workstation's display screen.
Various techniques have been developed for identifying the tool in a camera image. One such technique is described, for example, in Guo-Qing Wei, Klaus Arbter, and Gerd Hirzinger, “Real-Time Visual Servoing for Laparoscopic Surgery,” IEEE Engineering in Medicine and Biology, Vol. 16, No. 1, pp. 40-45, 1997, wherein a color marker is used to identify a tool in a camera image so that the camera may be automatically maneuvered so as to keep the tool in the center of its visual image. Another such mechanism is described, for example, in Xiaoli Zhang and Shahram Payandeh, “Application of Visual Tracking for Robot-Assisted Laparoscopic Surgery,” Journal of Robotics Systems, Vol. 19, No. 7, pp. 315-328, 2002, wherein a striped marker is used to identify a tool and its depth in a camera image so that the camera may be automatically maneuvered to visually track the tool.
These techniques, however, are not generally useful when the end effector is outside the endoscope's field of view, or when it is occluded by fluid or some object. In either case, such markers cannot be found and tool identification is not possible. Second, simple color markers and other orientation independent markers placed on the tool do not facilitate determining the orientation or pose of the tool. Third, computational complexity in identifying and tracking the tool markers may make real-time tracking difficult. In particular, increased time spent processing each captured frame of video will reduce the frequency at which video capture can occur, possibly resulting in abrupt transitions from one image to the next. Fourth, uncorrected camera calibration and/or other system measurement errors may result in errors when determining tool positions and orientations from the camera's visual images.